Papers with Retrieval augmented generation
TRAQ: Trustworthy Retrieval Augmented Question Answering via Conformal Prediction (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) often generate incorrect responses based on made-up facts, which are called hallucinations. |
| Approach: | They propose a framework that combines Retrieval Augmented Generation with conformal prediction to provide the first end-to-end statistical correctness guarantee for RAG. |
| Outcome: | The proposed framework reduces prediction set size by 16.2% on average compared to an ablation. |
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA (2024.emnlp-main)
Copied to clipboard
Minzheng Wang, Longze Chen, Fu Cheng, Shengyi Liao, Xinghua Zhang, Bingli Wu, Haiyang Yu, Nan Xu, Lei Zhang, Run Luo, Yunshui Li, Min Yang, Fei Huang, Yongbin Li
| Challenge: | Existing benchmarks for evaluating long-context language models employ irrelevant noise texts to artificially extend the length of test cases, diverging from the real-world scenarios of long-constituency applications. |
| Approach: | They propose a long-context benchmark, Loong, aligning with realistic scenarios through extended multi-document question answering (QA) . |
| Outcome: | The proposed model can scale up the context window of large language models to perform in-depth analysis of multiple long documents. |
Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation (2024.emnlp-main)
Copied to clipboard
Zhuohang Li, Jiaxin Zhang, Chao Yan, Kamalika Das, Sricharan Kumar, Murat Kantarcioglu, Bradley Malin
| Challenge: | Language models suffer from poor interpretability and transparency, as well as the intrinsic risk of hallucination and misinformation. |
| Approach: | They propose a statistical framework that assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge. |
| Outcome: | The proposed framework assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge. |